Traffic monitoring system and traffic monitoring method

ABSTRACT

The traffic monitoring system is provided with: a camera which captures an image of a monitoring area including a road and generates image data; a millimeter-wave radar which scans a scanning area included in the monitoring area and generates millimeter-wave data; and an information processing server which is connected to the camera and the millimeter-wave radar and acquires the image data and the millimeter-wave data. The information processing server is provided with: a data synchronization unit which synchronizes the image data with the millimeter-wave data so that the difference between a timing at which the image data is generated and a timing at which the millimeter-wave data is generated is equal to or smaller than a certain value; and a screen generation unit which associates the image data and the millimeter wave, which have been synchronized, with each other and generates a monitoring screen that indicates the road conditions.

TECHNICAL FIELD

The present disclosure relates to a traffic monitoring system and atraffic monitoring method.

BACKGROUND ART

Conventionally, traffic monitoring systems that monitor movement ofvehicles including bicycles and of pedestrians have been introduced onroads such as intersections. In traffic monitoring systems, it isdesirable that road conditions and/or a hazard alert be notifiedefficiently by detecting vehicles and pedestrians (hereafter, eachreferred to as a moving body) accurately and without omission.

For example, Patent Literature (hereinafter, referred to as “PTL”) 1discloses a method in which road condition images corresponding to eachpredetermined section of a road are generated based on images capturedby a monitoring camera, and are displayed with predetermined displaycolors.

Further, for example, PTL 2 discloses a device that measures runningvehicles by a millimeter-wave radar, and monitors vehicle traffic whileautomatically judging the number of lanes, each lane width, median stripwidths, and the like.

CITATION LIST Patent Literature PTL 1 Japanese Patent ApplicationLaid-Open No. 2004-102545 PTL 2 Japanese Patent Application Laid-OpenNo. 2007-257536 SUMMARY OF INVENTION Technical Problem

However, a traffic monitoring system in which two sensor devices, thatis, a camera and a radar, are combined and road conditions are monitoredaccurately and efficiently has not been sufficiently taken intoconsideration.

One non-limiting and exemplary embodiment of the present disclosurefacilitates providing a traffic monitoring system and a trafficmonitoring method in which two sensor devices, that is, a camera and aradar, are combined and road conditions are monitored accurately andefficiently.

A traffic monitoring system according to an embodiment of the presentdisclosure includes: a camera that captures a monitoring area includinga road, and generates image data; a millimeter-wave radar that scans ascanning area included in the monitoring area, and generatesmillimeter-wave data; and an information processing server that isconnected to the camera and the millimeter-wave radar, and acquires theimage data and the millimeter-wave data, wherein the informationprocessing server includes: a data synchronizer that synchronizes theimage data and the millimeter-wave data such that a difference between atiming at which the image data is generated and a timing at which themillimeter-wave data is generated is equal to or smaller than a certainvalue; and a screen generator that associates the image data and themillimeter-wave data which have been synchronized, with each other, andgenerates a monitoring screen that indicates a condition of the road.

A traffic monitoring method according to an embodiment of the presentdisclosure includes: acquiring image data from a camera, the image databeing generated by capturing a monitoring area including a road;acquiring millimeter-wave data from a millimeter-wave radar, themillimeter-wave data being generated by scanning a scanning areaincluded in the monitoring area; synchronizing the image data and themillimeter-wave data such that a difference between a timing at whichthe image data is generated and a timing at which the millimeter-wavedata is generated is equal to or smaller than a certain value; andassociating the image data and the millimeter-wave data which have beensynchronized, with each other, and generating a monitoring screen thatindicates a condition of the road.

It should be noted that general or specific embodiments may beimplemented as a system, an integrated circuit, a computer program or astorage medium, or may be implemented as any combination of a system, anapparatus, a method, an integrated circuit, a computer program, and astorage medium.

An embodiment of the present disclosure facilitates providing a trafficmonitoring system and a traffic monitoring method in which two sensordevices, that is, a camera and a radar, are combined and road conditionsare monitored accurately and efficiently.

Additional benefits and advantages of the disclosed embodiment willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates an example of a configuration of a traffic monitoringsystem according to an embodiment of the present disclosure;

FIG. 2 illustrates an example of a configuration of an informationprocessing server according to the embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating an example of processing of theinformation processing server according to the embodiment of the presentdisclosure;

FIG. 4 illustrates display example 1 of a display screen in theembodiment of the present disclosure;

FIG. 5 illustrates display example 2 of the display screen in theembodiment of the present disclosure;

FIG. 6 illustrates display example 3 of the display screen in theembodiment of the present disclosure;

FIG. 7 illustrates display example 4 of the display screen in theembodiment of the present disclosure; and

FIG. 8 illustrates display example 5 of the display screen in theembodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the present disclosure will be describedin detail with reference to the accompanying drawings as appropriate.However, more detailed description than necessary may be omitted. Forexample, detailed descriptions of already well-known matters andrepeated descriptions for substantially the same configuration may beomitted. This is to prevent the following description from becomingunnecessarily redundant and to facilitate understanding by those skilledin the art.

Note that, the accompanying drawings and the following description areprovided so that those skilled in the art understand the presentembodiment sufficiently, and are not intended to limit the subjectmatters recited in the claims.

EMBODIMENT

<Configuration of Traffic Monitoring System>

FIG. 1 illustrates an example of a configuration of traffic monitoringsystem 1 according to the present embodiment. Traffic monitoring system1 includes a plurality of cameras 2, a plurality of millimeter-waveradars 3, information processing server (control apparatus) 4, and aremote monitoring personal computer (PC) 5. In the present embodiment, atraffic monitoring system in which traffic of a road where vehicles,pedestrians, or the like pass is monitored will be described as anexample.

Camera 2 and millimeter-wave radar 3 are connected to informationprocessing server 4 through network N1. Network N1 may be a wirelessnetwork, a wired network, or a combination thereof.

Information processing server 4 is connected to remote monitoring PC 5through network N2. Network N2 may be a wireless network, a wirednetwork, or a combination thereof.

Camera 2 is provided, for example, above a structure (for example, apole on which a road sign is installed) on a periphery of the road.Camera 2 captures an area on the periphery of the road, which includesthe road. Note that, the area on the periphery of the road, which is ina capturing range of camera 2 and includes the road, may also bedescribed as a monitoring area.

Camera 2 generates data of an image (image data) captured thereby, andtransmits the captured image data to information processing server 4.The image data to be transmitted includes time information indicating atiming (for example, a time) at which the image data is generated. Notethat, the time indicated by the time information may also be a time whencamera 2 performs capturing. Further, for example, camera 2 may alsotransmit the image data in frame units to information processing server4. Further, the image captured by camera 2 may also be described as acamera image. Further, for example, a coordinate system defininghorizontal and vertical directions of the camera image may be describedas a camera coordinate system. The camera coordinate system is defined,for example, based on a position where camera 2 is installed, anorientation of camera 2, and a viewing angle of the camera.

Millimeter-wave radar 3 is provided, for example, above a structure on aperiphery of the road. Millimeter-wave radar 3 transmits a radar signalin the millimeter-wave band to the periphery of the road, and receives areflected signal that is the radar signal reflected by an object on theperiphery of the road. Note that, millimeter-wave radar 3 scans theperiphery of the road by sequentially transmitting radar signals in aplurality of directions. A range in which millimeter-wave radar 3 scansradar signals may also be described as a scanning area.

Millimeter-wave radar 3 generates, for example, millimeter-wave databased on information on at least one of a time between a time when aradar signal is transmitted and a time when a reflected signal isreceived, a direction in which a radar signal is transmitted, areception intensity of a reflected signal, and a Doppler frequency of areflected signal.

Millimeter-wave data includes, for example, information indicating anobject on the periphery of the road, which has reflected a radar signal(hereinafter, the object will be referred to as a reflection object).For example, millimeter-wave data is data including a set of pointsindicating a position of a reflection object (hereinafter, each of thepoints will be referred to as a reflection point) in a coordinate systemdefined by using a position of millimeter-wave radar 3 as a reference.The coordinate system defined by using a position of millimeter-waveradar 3 as a reference is, for example, a polar coordinate system inwhich each point on a plane is determined by a distance from a positionof millimeter-wave radar 3 and an azimuth at which millimeter-wave radar3 performs scanning. Note that, the coordinate system defined by using aposition of millimeter-wave radar 3 as a reference may also be describedas a millimeter-wave radar coordinate system. The millimeter-wave radarcoordinate system is defined, for example, based on a positionmillimeter-wave radar 3, an orientation of millimeter-wave radar 3, anda scanning area (detection range) of millimeter-wave radar 3.

Note that the reflection point is not limited to one reflection pointwith respect to one reflection object. In millimeter-wave data, onereflection object may be represented by a plurality of reflectionpoints.

Further, examples of the reflection object include vehicles andpedestrians (hereinafter, each referred to as a moving body) that travelon the periphery of the road, and structures provided on the peripheryof the road (road signs, signals, or the like; hereinafter, eachstructure will be referred to as a stationary object). Millimeter-wavedata may include a reflection point indicating a position of a movingbody, and a reflection point indicating a position of a stationaryobject.

Millimeter-wave radar 3 transmits millimeter-wave data to informationprocessing server 4. The millimeter-wave data to be transmitted includestime information indicating a timing (for example, a time) at which themillimeter-wave data is generated. Note that, the time indicated by thetime information may also be a time when a radar signal for generatingthe millimeter-wave data is transmitted, or a time when a radar signalis received. Further, millimeter-wave radar 3 may scan a scanning areaat a set period and transmit the millimeter-wave data to informationprocessing server 4.

The timing at which camera 2 generates image data and the timing atwhich millimeter-wave radar 3 generates millimeter-wave data may notcoincide, and may be different from each other. Further, the timing atwhich camera 2 transmits image data and the timing at whichmillimeter-wave radar 3 transmits millimeter-wave data may not coincide,and may be different from each other. For example, a frequency at whichcamera 2 generates image data is higher than a frequency at whichmillimeter-wave radar 3 generates millimeter-wave data.

Note that, camera 2 and millimeter-wave radar 3 may be installed in thesame structure or in structures different from each other. Further,camera 2 and millimeter-wave radar 3 may be provided in the same housingor in separate housings.

Further, methods for installing camera 2 and millimeter-wave radar 3,places where camera 2 and millimeter-wave radar 3 are installed, and arelative positional relationship between camera 2 and millimeter-waveradar 3 are not limited. Further, a positional relationship between themonitoring area of camera 2 and the scanning area of millimeter-waveradar 3 is not limited. In the present disclosure, camera 2 andmillimeter-wave radar 3 are preferably installed such that the scanningarea of millimeter-wave radar 3 is included in the monitoring area ofcamera 2.

For example, at least one camera 2 and at least one millimeter-waveradar 3 are paired and provided at one point where monitoring isperformed (hereinafter, the point will be referred to as a monitoringpoint). Note that, at one monitoring point, two or more cameras 2 may beprovided and/or two or more millimeter-wave radars 3 may be provided.

Information processing server 4 is connected to camera 2 andmillimeter-wave radar 3, which are provided at each of a plurality ofmonitoring points, through network N1. Information processing server 4acquires image data from camera 2, and acquires millimeter-wave datafrom millimeter-wave radar 3. Further, information processing server 4generates a monitoring screen related to road conditions at themonitoring point based on the image data and the millimeter-wave data.Information processing server 4 transmits data of the generatedmonitoring screen to remote monitoring PC 5 through network N2.

Note that, information processing server 4 may receive instructioninformation including a setting related to the monitoring screen and/oran instruction related to a monitoring point, from remote monitoring PC5. In this case, information processing server 4 generates a monitoringscreen based on the instruction information.

Remote monitoring PC 5 receives the data of the monitoring screen frominformation processing server 4 through network N2. Remote monitoring PC5 processes the data of the monitoring screen, and displays themonitoring screen on a display (not illustrated).

Note that, remote monitoring PC 5 may receive a setting related to themonitoring screen and/or an instruction related to a monitoring pointfrom a user through an operator (not illustrated), for example. In thiscase, remote monitoring PC 5 may transmit instruction informationincluding the setting related to the monitoring screen and/or theinstruction related to a monitoring point to information processingserver 4.

Note that, although FIG. 1 illustrates one remote monitoring PC5, theremay be a plurality of remote monitoring PCs 5. Further, although FIG. 1illustrates networks N1 and N2, networks N1 and N2 may be the samenetwork or different networks. Further, although an example has beendescribed in which information processing server 4 is connected tocamera 2 and millimeter-wave radar 3 through network N1, informationprocessing server 4 may be connected to camera 2 and/or millimeter-waveradar 3 directly by a wired or wireless connection, not through networkN1. Further, remote monitoring PC 5 may be connected to informationprocessing server 4 directly by a wired or wireless connection, notthrough network N2.

In traffic monitoring system 1 described above, in a case where a userwho performs monitoring inputs (or selects) a specific monitoring pointthrough the operator of remote monitoring PC 5, for example, remotemonitoring PC 5 transmits instruction information indicating the input(or selected) monitoring point to information processing server 4.

Information processing server 4 generates a monitoring screen related toroad conditions at the monitoring point indicated by the instructioninformation. Information processing server 4 then transmits data of thegenerated monitoring screen to remote monitoring PC 5 that is thetransmission source of the instruction information.

<Configuration of Information Processing Server>

Next, an example of a configuration of information processing server 4will be described. FIG. 2 illustrates an example of a configuration ofinformation processing server 4 according to the present embodiment.

For example, information processing server 4 includes communicator 41,data synchronizer 42, data accumulator 43, screen generator 44, and modesetter 45.

Communicator 41 is an interface for communication with camera 2 andmillimeter-wave radar 3 through network N1. Further, communicator 41 isan interface for communication with remote monitoring PC 5 throughnetwork N2.

Data synchronizer 42 acquires image data from camera 2 throughcommunicator 41. Further, data synchronizer 42 acquires millimeter-wavedata from millimeter-wave radar 3 through communicator 41.

Data synchronizer 42 synchronizes the image data and the millimeter-wavedata. For example, data synchronizer 42 adjusts a timing at which theimage data is generated and a timing at which the millimeter-wave datais generated, based on time information included in the image data andtime information included in the millimeter-wave data.

For example, for image data of one frame, data synchronizer 42 selectssuch millimeter-wave data that a time difference between a timeindicated by time information included in the image data and a timeindicated by time information included in the millimeter-wave data isequal to or smaller than a predetermined value, and associates theselected millimeter-wave data with the image data of the one frame.Alternatively, for image data of one frame, data synchronizer 42 mayselect millimeter-wave data including time information indicating a timeclosest to a time indicated by time information included in the imagedata, and associate the selected millimeter-wave data with the imagedata of the one frame.

Note that, for example, in a case where the frequency at which camera 2generates image data and a frequency at which millimeter-wave radar 3generates millimeter-wave data are different, data synchronizer 42 mayperform data synchronization so as to associate one image data with twodifferent millimeter-wave data. Alternatively, data synchronizer 42 mayperform data synchronization so as to associate one millimeter-wave datawith two different image data. Data synchronizer 42 associates the imagedata and the millimeter-wave data with each other, in which the timesindicated by each time information are included within a predeterminedtime.

Data synchronizer 42 outputs the synchronized data to data accumulator43. Further, data synchronizer 42 outputs the synchronized data toscreen generator 44.

Data accumulator 43 accumulates the image data and the millimeter-wavedata which have been synchronized by data synchronizer 42, inassociation with each other. Data accumulator 43 accumulates the imagedata and the millimeter-wave data in time series based on the timeinformation, for example. Further, data accumulator 43 may accumulateimage data and millimeter-wave data of each of the plurality ofmonitoring points.

Screen generator 44 acquires the image data and the millimeter-wave datawhich have been synchronized (whose timings have been adjusted), fromdata synchronizer 42. Further, screen generator 44 may acquireinstruction information including a setting related to the monitoringscreen and/or an instruction related to a monitoring point from remotemonitoring PC 5 through communicator 41.

Screen generator 44 then associates the image data and themillimeter-wave data which have been synchronized (whose timings havebeen adjusted), with each other, and generates a monitoring screen thatindicates road conditions.

For example, screen generator 44 performs signal processing (forexample, clustering processing) to millimeter-wave data ofmillimeter-wave radar 3 provided at a monitoring point indicated byinstruction information, and estimates a region corresponding to areflection object. The estimated region is an example of millimeter-wavedata information.

Further, screen generator 44 performs conversion such thatmillimeter-wave data corresponding to a scanning area corresponds to amonitoring area. For example, screen generator 44 may perform coordinateconversion processing in which millimeter-wave data information definedby the millimeter-wave radar coordinate system is converted into that ofthe camera coordinate system. Screen generator 44 superimposesmillimeter-wave data information after the coordinate conversion on animage indicated by image data.

Further, screen generator 44 may perform signal processing in accordancewith a mode instructed by mode setter 45, and generate a monitoringscreen in accordance with the mode. Note that, examples of the mode andthe monitoring screen in accordance with the mode will be describedlater.

Screen generator 44 then transmits data of the generated monitoringscreen to remote monitoring PC 5. Note that, screen generator 44 maytransmit the millimeter-wave data used for generating the data of themonitoring screen to remote monitoring PC5.

Note that, screen generator 44 may also generate a monitoring screen byusing data accumulated in data accumulator 43.

Mode setter 45 acquires instruction information through communicator 41.

Mode setter 45 instructs a mode related to the monitoring screen, whichis included in the instruction information, to screen generator 44.

<Processing Flow in Information Processing Server>

Next, an example of a processing flow to be executed in informationprocessing server 4 will be described with reference to FIG. 3.

FIG. 3 is a flowchart illustrating an example of processing ofinformation processing server 4 according to the present embodiment.Note that, the flowchart illustrated in FIG. 3 indicates processing todata acquired from camera 2 and millimeter-wave radar 3 provided at onemonitoring point. Information processing server 4 may simultaneouslyperform the processing illustrated in FIG. 3 to data acquired fromcamera 2 and millimeter-wave radar 3 provided at each monitoring point.

Data synchronizer 42 acquires image data from camera 2 (S101).

Data synchronizer 42 acquires millimeter-wave data from millimeter-waveradar 3 (S102).

Note that, the order of S101 and S102 is not limited thereto. Further,the processing of S101 is executed each time camera 2 transmits imagedata, and the processing of S102 is executed each time millimeter-waveradar 3 transmits millimeter-wave data.

Data synchronizer 42 performs data synchronization processing in whichthe image data and the millimeter-wave data are synchronized (S103).

Data synchronizer 42 performs data accumulation processing in which thesynchronized data are accumulated in data accumulator 43 (S104).

Screen generator 44 performs clustering processing to themillimeter-wave data (S105).

The clustering processing is processing in which a plurality ofreflection points included in millimeter-wave data and corresponding toone reflection object are grouped. Information on the reflection object,such as the size, shape, and color of the reflection object, may also beacquired by the grouping processing. Note that, a region including agroup of a plurality of reflection points corresponding to onereflection object after the clustering processing will be describedhereinafter as a reflection region.

Note that, in the clustering processing, screen generator 44 may alsouse millimeter-wave data before current millimeter-wave data, which isaccumulated in data accumulator 43. By using the millimeter-wave databefore the current millimeter-wave data and performing time-seriesprocessing, screen generator 44 may, for example, distinguish between areflection point corresponding to a moving body and a reflection pointcorresponding to a stationary object.

Next, screen generator 44 performs data processing in accordance with aset mode (S106).

For example, in a case where the set mode is a mode in which a movingdirection of a moving object is monitored (for example, a wrong-waytraveling detection mode to be described later), screen generator 44performs processing of tracking the moving object. For example, screengenerator 44 performs processing in which a moving direction of areflection region corresponding to the moving object is determined as anexample of the tracking processing.

Further, for example, in a case where the set mode is a mode in which atype of a reflection object is identified (for example, an objectidentification mode to be described later), screen generator 44identifies the type of the reflection object by using a receptionintensity of a reflected signal, a size of a reflection region, and/orthe like that are indicated by millimeter-wave data.

Further, for example, in a case where the set mode is a mode in which aretention condition of a reflection object is determined (for example, aheat map mode to be described later), screen generator 44 calculates thenumber of times (frequency) when the reflection object is detected in ascanning area for each partial range by using millimeter-wave datawithin a predetermined time accumulated in data accumulator 43.

Note that, the data processing in accordance with the set mode is notlimited to the examples described above.

Next, screen generator 44 performs processing of generating a monitoringscreen in which the image data and the millimeter-wave data have beenassociated with each other (S107).

For example, screen generator 44 performs processing of superimposingthe millimeter-wave data on a camera image indicated by the image data.In this processing, screen generator 44 may perform coordinateconversion of millimeter-wave data information such that the cameracoordinate system and the millimeter-wave radar coordinate system arealigned.

Screen generator 44 performs processing of transmitting data of thegenerated monitoring screen (S108).

Next, display examples of monitoring screens generated in informationprocessing server 4 and displayed on the display of remote monitoring PC5 will be described.

Display Example 1

Display example 1 is an example of screens displayed on the display ofremote monitoring PC 5 in a case where a user performing monitoringselects a monitoring point of an intersection through the operator ofremote monitoring PC 5.

FIG. 4 illustrates display example 1 of a display screen in the presentembodiment. Display example 1 illustrated in FIG. 4 includes amonitoring screen generated based on data acquired from camera 2 that isthe monitoring point and is provided in an obliquely upward directionwhen viewed from a road surface of an intersection, and millimeter-waveradar 3 provided at substantially the same position as that of camera 2.

In an upper portion of FIG. 4, a “wrong-way traveling detection” button,an “object identification” button, a “heat map” button, and a “setting”button are illustrated. Each button is depressed by a user in a casewhere a setting related to the monitoring screen, for example, a moderelated to the monitoring screen is set.

In a lower portion of FIG. 4, region R1 in which millimeter-wave data isdisplayed is illustrated. Further, an example of the monitoring screenis illustrated in region R2.

For example, region R1 indicates a plurality of grids delimited by aplurality of vertical lines and a plurality of horizontal lines in whicha position of millimeter-wave radar 3 is used as a reference (origin).Further, region R1 indicates area A1 indicating a scanning area ofmillimeter-wave radar 3, and reflection points in area A1. Note that,the display of region R1 may be omitted as appropriate by a setting of auser, for example.

For example, on the monitoring screen of region R2, millimeter-wave dataacquired by scanning by millimeter-wave radar 3 is superimposed on acamera image.

Note that, screen generator 44 may perform coordinate conversionprocessing in a case where the millimeter-wave data is superimposed onthe camera image. For example, screen generator 44 converts area A1 andthe reflection points in area A1, which are defined by themillimeter-wave radar coordinate system, into those in the cameracoordinate system. On the monitoring screen of region R2, themillimeter-wave data converted to that of the coordinate system of themonitoring area of camera 2 is superimposed on the camera image.

For example, area A1 of region R1 defined by the millimeter-wave radarcoordinate system corresponds to an area sandwiched by lines L1 and L2on the monitoring screen of region R2 defined by the camera coordinatesystem.

Note that, the area sandwiched by lines L1 and L2 of region R2 indicatesthe scanning area of millimeter-wave radar 3 on the camera image. Thearea sandwiched by lines L1 and L2 may be indicated with a differentaspect (for example, a different color) from the camera image such thatthe camera image is transmitted, for example. Further, the areasandwiched by lines L1 and L2 may be omitted as appropriate.

In the area sandwiched by lines L1 and L2 of region R2, frames r1 to r6indicating reflection regions corresponding to reflection objectsdetected based on the millimeter-wave data are indicated. Thus, sincethe monitoring screen of region R2 indicates reflection regionscorresponding to reflection objects by using frames on the camera image,the reflection objects present in the monitoring area and detected bymillimeter-wave radar 3 can be visually confirmed by a user.

Further, frames r1 to r4 are indicated with a different aspect (forexample, a different color) from that of frames r5 and r6. The aspectsof frames vary depending on whether reflection objects detected by usingmillimeter wave data are moving objects or stationary objects. Changingthe aspects of frames indicating reflection objects enables a user tovisually confirm information on whether the reflection objects detectedby millimeter-wave radar 3 are moving objects or stationary objects.

For example, frames r1 to r4 indicate each a moving object (a vehicle ora pedestrian), and frames r5 and r6 indicate each a stationary object (astructure provided on a roadside of the road).

Note that, the “wrong-way traveling detection” button, the “objectidentification” button, the “heat map” button, and the “setting” buttonin the upper portion of FIG. 4 correspond to a wrong-way travelingdetection mode, an object identification mode, a heat map mode, and asetting mode, respectively. A user may operate the operator of remotemonitoring PC 5 and depress any one of the buttons. In this case, remotemonitoring PC 5 transmits to information processing server 4 that themode corresponding to a depressed button has been set.

Next, a display example in a case where information processing server 4changes the mode will be described.

Display Example 2

Display example 2 is an example of a screen displayed on the display ofremote monitoring PC 5 in a case where the user performing monitoringdepresses the “wrong-way traveling detection” button through theoperator of remote monitoring PC 5 and the wrong-way traveling detectionmode is set.

FIG. 5 illustrates display example 2 of the display screen in thepresent embodiment. Display example 2 illustrated in FIG. 5 includes amonitoring screen generated based on data acquired from camera 2 that isthe monitoring point and is provided in an upward direction of a roadsurface of a road having a total of four lanes, with two lanes on eachside, and millimeter-wave radar 3 provided at substantially the sameposition as that of camera 2.

In an upper portion of FIG. 5, the “wrong-way traveling detection”button, the “object identification” button, the “heat map” button, andthe “setting” button are illustrated. Since the wrong-way travelingdetection mode is set in FIG. 5, FIG. 5 is in a state in which the“wrong-way traveling detection” button is depressed.

Region R3 is a monitor screen displayed in a case where the wrong-waytraveling detection mode is set. In region R3, millimeter-wave data issuperimposed on a camera image.

Note that, as in the example of FIG. 4, screen generator 44 may performcoordinate conversion processing in a case where the millimeter-wavedata is superimposed on the camera image. Further, in a case where thewrong-way traveling detection mode is set, screen generator 44 performsprocessing in which wrong-way traveling of a moving object (for example,a vehicle) is detected.

For example, in the processing of detecting wrong-way traveling, screengenerator 44 detects a moving object present in a scanning area by usingmillimeter-wave data. Screen generator 44 then detects a movingdirection of the detected moving object. A method for detecting of amoving direction of a moving body may be, for example, a method using aDoppler shift or a method in which a position of the moving body at acurrent point in time is compared with that at a point in time beforethe current point in time. A position of a moving object at a point intime before a current point in time may be detected, for example, byusing past millimeter-wave data accumulated in data accumulator 43.

Screen generator 44 may also superimpose frames (frames r7 and r8 inFIG. 5) indicating positions of detected moving objects and arrows(arrows x7 and x8 in FIG. 5) indicating moving directions of thedetected moving objects on the camera image. The frames indicating thepositions of the detected moving objects and the arrows indicating themoving directions of the detected moving objects represent an example ofmillimeter-wave data information.

Screen generator 44 then determines whether or not the moving directionsof the detected moving objects are different from a traveling direction(hereinafter, referred to as a forward direction) defined for the roadincluded in the monitoring area.

Note that, basically, when positions and orientations of camera 2 andmillimeter-wave radar 3 are determined and a monitoring area (and ascanning area) is determined, the forward direction is uniquelydetermined. Accordingly, the forward direction may be set in advance,for example, in accordance with positions where camera 2 andmillimeter-wave radar 3 are installed and orientations thereof.Alternatively, screen generator 44 may also determine the forwarddirection by monitoring moving directions of a plurality of movingobjects passing through a monitoring area for a predetermined time.

Note that, FIG. 5 illustrates arrow x0 indicating the forward directionfor convenience of description.

In a case where a moving direction of a detected moving object isdifferent from the forward direction, screen generator 44 determinesthat the detected moving object travels in the wrong direction.

For example, in FIG. 5, arrow x7 of a moving body surrounded by frame r7is the same direction as arrow x0 that is the forward direction.Accordingly, it is determined that the moving body surrounded by framer7 does not travel in the wrong direction. On the other hand, arrow x8of a moving body surrounded by frame r8 is a direction opposite to arrowx0 that is the forward direction. Accordingly, it is determined that themoving body surrounded by frame r8 travels in the wrong direction.

Note that, in a case where it is determined whether or not wrong-waytraveling occurs, screen generator 44 may define scales z finelysectioning the road as illustrated in FIG. 5, and determine the movingdirection in each scale z. Determining the moving direction in aplurality of scales makes it possible to restrain a determination error.

In a case where a moving object traveling in the wrong direction isdetected, screen generator 44 transmits alarm information indicatingthat the moving object traveling in the wrong direction has beendetected.

Remote monitoring PC 5 that receives the alarm information notifies thedisplay of an alarm. An alarm notification method may be, for example, amethod using symbols and/or character information or a method in whichat least some colors of the display are changed. Alternatively, remotemonitoring PC 5 may notify the alarm by using audio.

Note that, screen generator 44 may capture a moving body traveling inthe wrong direction and determine a wrong-way traveling situation of themoving body by using millimeter-wave data before current millimeter-wavedata. Then, screen generator 44 may add information indicating thewrong-way traveling situation of the moving body to the alarminformation. The wrong-way traveling situation of the moving bodyrepresents, for example, whether or not the wrong-way traveling isintentional or whether or not the wrong-way traveling is caused by anoperational error by the driver.

For example, in a case where a moving object continues wrong-waytraveling for a relatively long time, screen generator 44 determinesthat the wrong-way traveling is intentional. Alternatively, in a casewhere a moving object suddenly reverses the traveling direction, thatis, performs a so-called U-turn, screen generator 44 determines that thewrong-way traveling is intentional. Alternatively, in a case where amoving object temporarily travels in the wrong direction and thenchanges the traveling direction to the forward direction, screengenerator 44 determines that the wrong-way traveling is caused by anoperational error (for example, “inadvertence”) by the driver.

For example, a situation is assumed in which, in a case where a vehiclethat has entered a rest facility such as a service area and a parkingarea on an expressway merges into a main lane on the expressway, thevehicle travels in the wrong direction due to an inadvertent operationby the driver, the driver notices the wrong-way traveling on the way,and changes the direction to the forward direction. Such a situation maybe determined as a temporary wrong-way traveling. Note that, there mayalso be a situation in which a driver is unconscious and continueswrong-way traveling without noticing it. Such a situation may bedetermined as intentional wrong-way traveling.

Remote monitoring PC5 that has received alarm information may change analarm to be notified to the display in accordance with informationindicating a wrong-way traveling situation of a moving body. Note that,in a case where wrong-way traveling is determined on the monitoringscreen, screen generator 44 may perform a highlighted display, forexample, changing at least one of the thickness, type, color tone,brightness, and contrast of the frame line of frame r8, or blinking anddisplaying frame r8.

Further, information processing server 4 may be provided with an alarmcontroller that controls an external equipment and issues an alarm to amoving object traveling in the wrong direction and/or a periphery of themoving object traveling in the wrong direction based on alarminformation acquired from screen generator 44.

The alarm controller is provided, for example, on a periphery of amonitoring point, controls at least one of an alarm lamp, a lightemitting diode (LED) lighting pole, and a display board (guide board)that are connected to network N1, and issues an alarm to the peripheryof the monitoring point that there is a moving object traveling in thewrong direction. For example, the alarm controller controls the color,brightness, and interval between lighting and lighting-out of the alarmlamp and/or the LED lighting pole, and notifies the periphery of themonitoring point of an abnormality.

Thus, by superimposing a result of detecting wrong-way traveling byusing millimeter-wave data on a camera image, the wrong-way travelingdetection mode makes it possible to detect wrong-way traveling whosedetection may be delayed with a camera image, and to assist trafficmonitoring by a user's vision efficiently.

Display Example 3

Display example 3 is an example of a screen displayed on the display ofremote monitoring PC 5 in a case where the user performing monitoringdepresses the “object identification” button through the operator ofremote monitoring PC 5 and the object identification mode is set.

FIG. 6 illustrates display example 3 of the display screen in thepresent embodiment. Display example 3 illustrated in FIG. 6 includes amonitoring screen generated based on data acquired from camera 2 that isthe monitoring point and is provided in an obliquely upward directionwhen viewed from a road surface of an intersection, and millimeter-waveradar 3 provided at substantially the same position as that of camera 2.

In an upper portion of FIG. 6, the “wrong-way traveling detection”button, the “object identification” button, the “heat map” button, andthe “setting” button are illustrated. Since the object identificationmode is set in FIG. 6, FIG. 6 is in a state in which the “objectidentification” button is depressed.

Region R4 is a monitoring screen set to the object identification mode.In region R4, millimeter-wave data is superimposed on a camera image.

Note that, as in the example of FIG. 4, screen generator 44 may performcoordinate conversion processing in a case where the millimeter-wavedata is superimposed on the camera image. Further, in a case where theobject identification mode is set, screen generator 44 performsprocessing in which an object detected in the millimeter-wave data isidentified.

For example, screen generator 44 detects an object present in themonitoring area by using millimeter-wave data. The object to be detectedmay be a moving object or a stationary object. Screen generator 44 thendetects a feature whereby the detected object is identified. The featurewhereby an object is identified may be, for example, at least one of thetype (for example, extra-large vehicle, large vehicle, ordinary vehicle,two-wheeler and person), size (for example, width and height), color,and shape of the object.

A method for detecting a feature of an object may be, for example, apublicly-known method based on information on at least of a receptionintensity of a reflected signal and a Doppler frequency of a reflectedsignal, or other information.

Screen generator 44 may superimpose a frame indicating a position of adetected object and information indicating a feature of the detectedobject on the camera image. For example, it is possible to identify thetype of person, bicycle, bike, ordinary vehicle, large vehicle or thelike as a moving object.

The information indicating a feature of an object may be, for example,character information. Alternatively, a feature of an object may beindicated by changing the type and/or color of a frame indicating aposition of the object depending on the feature of the object.

Further, information indicating probability (reliability) of a featurefor each type of an object may also be superimposed.

For example, in FIG. 6, frames r9 and r10 indicating positions ofdetected vehicles are indicated with a different aspect (for example, adifferent color) from that of frames r11 and r12 indicating positions ofdetected pedestrians. It is then indicated with character informationthat the type of the vehicle indicated by frame r9 is “ordinaryvehicle”. In addition, it is indicated with character information thatthe probability of those features is “0.95”. This numerical valueindicates, for example, that a reflection object indicated by frame r9is an ordinary vehicle with 95% reliability.

Thus, by superimposing information on an identification resultindicating a feature of an object detected by using millimeter-wave dataon a camera image, the object identification mode enables a user toconfirm the feature of the object, which is difficult to confirm withthe camera image, and to monitor traffic monitoring visually. Forexample, it is possible to confirm a feature of an object even in a casewhere a camera image is not clear due to weather and/or date and time.Note that, in a situation in which weather or the like does not affect,information such as black and white may also be added as colorinformation on an object by using an analysis result of a camera imageas illustrated in FIG. 6.

Display Example 4

Display example 4 is an example of a screen displayed on the display ofremote monitoring PC 5 in a case where the user performing monitoringdepresses the “heat map” button through the operator of remotemonitoring PC 5 and the heat map mode is set.

FIG. 7 illustrates display example 4 of the display screen in thepresent embodiment. Display example 4 illustrated in FIG. 7 includes amonitoring screen generated based on data acquired from camera 2 that isthe monitoring point and is provided in an obliquely upward directionwhen viewed from a road surface of an intersection, and millimeter-waveradar 3 provided at substantially the same position as that of camera 2.

In an upper portion of FIG. 7, the “wrong-way traveling detection”button, the “object identification” button, the “heat map” button, andthe “setting” button are illustrated. Since the heat map mode is set inFIG. 7, FIG. 7 is in a state in which the “heat map” button isdepressed.

Region R5 is a monitor screen set to the heat map mode. In region R5,millimeter-wave data is superimposed on a camera image.

Note that, as in the example of FIG. 4, screen generator 44 may performcoordinate conversion processing in a case where the millimeter-wavedata is superimposed on the camera image. Further, in a case where theheat map mode is set, screen generator 44 performs processing in which aretention condition of an object is determined by using millimeter-wavedata before current millimeter-wave data and heat map informationindicating the retention condition is generated.

For example, by using millimeter-wave data acquired at a predeterminedtime before a current point in time, screen generator 44 counts thenumber of reflection objects detected at a predetermined time in each ofdetection rectangular frames obtained by dividing a monitoring area intoa plurality of grids. Screen generator 44 then classifies the detectionrectangular frames in accordance with the number of detected reflectionobjects. At this time, the detection rectangular frames may also beclassified in accordance with the type (for example, vehicle orpedestrian) and number of the detected reflection objects.

Then, screen generator 44 may also superimpose the heat map informationindicating the classified detection rectangular frames on the cameraimage.

The heat map information on the indicating the classified detectionrectangular frames is, for example, a difference in aspect such as acolor difference. For example, screen generator 44 may superimposecolors corresponding to the number of detected reflection objects withan aspect of making the colors transparent on the camera image.

For example, FIG. 7 illustrates information indicating detectionrectangular frames classified in accordance with the number of detectedreflection objects with four color differences of “Lv1” to “Lv4”. Forexample, regions of Lv2 to Lv4 in FIG. 7 correspond to regions of aroadway. Further, a region of Lv4 falls under a region corresponding toa crosswalk where a frequency at which both vehicles and pedestrianspass is high, or a region corresponding to a stop line where the numberof vehicle stops is high. Further, a comparison between the regions ofLv4 and Lv3 and the region of Lv2 clarifies that the intersection at themonitoring point is an intersection where left turns are few.

Thus, superimposing heat map information on a camera image makes itpossible to estimate a road structure (for example, the number of lanes,the lane type (for example, right-turn dedicated lane), and the laneshape) at a monitoring point. For example, it is possible to estimatethat a region where the number of vehicle detections is high is aroadway, that a region where the number of pedestrian detections is highis a sidewalk, and that a region where the number of both vehicle andpedestrian detections is high is a crosswalk. Further, it is possible toestimate that, in a region of a roadway, a position where the number ofvehicle detections is relatively high is a periphery of a stop line.

Thereby, a road structure of a monitoring area can be accuratelyestimated. Accordingly, it is possible to perform a display with highvisual effect on a user.

Further, since a road structure of a monitoring area can be accuratelyestimated, it is possible to accurately predict a situation such asrunning-out of a pedestrian and/or a vehicle and a two-wheeler'sslipping-through, and to issue an effective alarm at a monitoring point.

Further, since a road structure of a monitoring area can be accuratelyestimated, it is possible to reduce labor for setting a road structurewhen camera 2 and millimeter-wave radar 3 are installed, so that thesetting can be simplified.

Note that, display examples 2 to 4 (FIGS. 5 to 7) described above haveindicated examples in which one monitoring screen is displayed. Thepresent disclosure is not limited thereto. For example, millimeter-wavedata acquirable by scanning by millimeter-wave radar 3 may be displayedin FIGS. 5 to 7 as in region R1 of FIG. 4.

Display Example 5

Display example 5 is an example of a screen displayed on the display ofremote monitoring PC 5 in a case where the user performing monitoringdepresses the “setting” button through the operator of remote monitoringPC 5 and the setting mode is set.

FIG. 8 illustrates display example 5 of the display screen in thepresent embodiment. In an upper portion of FIG. 8, the “wrong-waytraveling detection” button, the “object identification” button, the“heat map” button, and the “setting” button are illustrated. Since thesetting mode is set in FIG. 8, FIG. 8 is in a state in which the“setting” button is depressed.

Further, buttons of “maintenance mode”, “weather estimation mode”, and“night mode” that can be set in the setting mode are displayed on thedisplay screen of FIG. 8. The user performing monitoring depresses adisplayed button, and selects a different mode. Hereinafter, each modewill be described.

“Maintenance Mode”

In the maintenance mode, a monitoring point is selected, and therebyinformation processing server 4 performs a fixed point observation ofcamera 2 and/or millimeter-wave radar 3 provided at the monitoringpoint. In a case where an abnormality of the monitoring point isdetected by the fixed point observation, information processing server 4adjusts parameters of camera 2 (for example, the viewing angle,resolution, and exposure of camera 2) and/or parameters ofmillimeter-wave radar 3 (for example, resolution, transmission output,and reception sensitivity). In a case where an abnormality is detectedeven when the parameter adjustment is repeated, information processingserver 4 notifies remote monitoring PC 5 of information informing theabnormality. Remote monitoring PC 5 that has received the notificationdisplays the information informing the abnormality on the display.

“Weather Estimation Mode”

In the weather estimation mode, information processing server 4estimates weather by using millimeter-wave data. For example,information processing server 4 may detect attenuation of a receptionintensity of a reflected signal and/or a variation in a Dopplerfrequency of a reflected signal indicated by the millimeter-wave data inaccordance with the difference in weather, and estimate weather based ona detected amount.

“Night Mode”

In the nighttime mode, information processing server 4 does notsuperimpose millimeter-wave data on a camera image, but mainly displaysa display based on millimeter-wave data. Thus, in a case where an imagecaptured by camera 2 at night or the like is not clear and an object isdifficult to identify, it is possible to avoid a display that is ratherdifficult to see when superimposition is performed.

Note that, the modes that can be set in the setting mode may includemodes indicating the following examples.

“Traffic Flow Count Mode”

In the traffic flow count mode, by using millimeter-wave data acquiredat a predetermined time before a current point in time, screen generator44 counts the number of vehicles detected at a predetermined time ineach of detection rectangular frames obtained by dividing a monitoringarea into a plurality of grids. Screen generator 44 then generatesinformation on traffic flow in accordance with the number of detectedobjects, and superimposes the information on a camera image.

For example, in the traffic flow count mode, screen generator 44statistically calculates information such as vehicle-to-vehicledistances, speeds, and vehicular lengths in accordance with the numberof detected vehicles. Screen generator 44 then estimates a degree ofcongestion in a monitoring area.

Note that, screen generator 44 may also perform the estimation for eachtime zone and/or for each date. The time zones and/or dates for whichthe estimation is performed may be designated by a user through remotemonitoring PC5.

Further, screen generator 44 may also estimate a degree of congestionfor each type of detected objects. For example, screen generator 44 mayestimate a degree of congestion for each type of vehicles, such asordinary vehicle, large vehicle, and two-wheeler.

“Intersection Monitoring Mode”

In the intersection monitoring mode, information processing server 4detects an event likely to occur at an intersection by using camera 2and millimeter-wave radar 3 provided on a periphery of the intersection,and issues an alarm in accordance with a detection result.

For example, to prevent a collision when a vehicle turns left,information processing server 4 confirms, in a case where a vehicleturning left is detected, whether or not there is a moving object (forexample, a two-wheeler) on a periphery of the vehicle turning left. In acase where there is a moving object on the periphery of the vehicleturning left, information processing server 4 then issues an alarm (forexample, an alarm against running-out) to the two-wheeler and aperiphery thereof.

Further, for example, to detect a two-wheeler's slipping-through,information processing server 4 prioritizes and detects a two-wheeler byusing millimeter-wave data. Information processing server 4 then tracksa travel path of the detected two-wheeler. In a case where thetwo-wheeler slips through between ordinary vehicles or the like,information processing server 4 issues an alarm (for example, an alarmagainst running-out) to the two-wheeler and a periphery thereof.

“Animal Detection Mode”

In the animal detection mode, information processing server 4 detectswhether or not there is an animal on a road or on a periphery of theroad based on millimeter-wave data. For example, information processingserver 4 extracts information peculiar to a case where an animal is areflection object from reflected signal information included inmillimeter-wave data, and detects whether or not there is an animal.

Note that, in the present embodiment, a traffic monitoring system inwhich traffic of a road where vehicles, pedestrians, or the like pass ismonitored has been described as an example. The present disclosure isnot limited thereto. For example, it may be a traffic monitoring systemin which railways and railway users are monitored. For example, in atraffic monitoring system in which railway traffic is monitored, camera2 and millimeter-wave radar 3 may be provided at locations looking overa platform and a track and/or locations closer to a track below aplatform than to the platform. Further, in the traffic monitoring systemin which railway traffic is monitored, a “fall detection mode” or thelike may be included in settable modes.

“Fall Detection Mode”

In the fall detection mode, in a case where camera 2 and/ormillimeter-wave radar 3 provided above a platform (hereinafter, referredto as “camera/radar on platform”) detect that a person walking on theplatform or an article (a portable article such as an umbrella or apurse) falls below the platform, camera 2 and/or millimeter-wave radar 3provided below the platform (hereinafter, referred to as “camera/radarbelow platform”) start detecting. Then, data of the camera/radar onplatform and data of the camera/radar below platform after a point intime when the fall below the platform is detected are stored inassociation with each other.

Further, in a case where the radar below platform does not detect amoving object, the camera below platform may extract a stationaryobject.

As described above, in the present embodiment, a traffic monitoringsystem has been described which includes an information processingserver that synchronizes image data captured by a camera andmillimeter-wave data acquired by a millimeter-wave radar, associates theimage data and the millimeter-wave data which have been synchronized,with each other, and generates a monitoring screen indicating roadconditions. According to the present embodiment, the data acquirablefrom two sensor devices, that is, a camera and a millimeter-wave radar,are synchronized, and a monitoring screen in which the synchronized dataare associated with each other is generated, so that road conditions canbe monitored accurately and efficiently.

For example, a camera image acquirable from data of a camera can providea user performing monitoring with visually effective information, anddata of a millimeter-wave radar can provide the user performingmonitoring with detailed information that cannot be acquired from thecamera image. Accordingly, the present embodiment makes it possible tomonitor road conditions accurately and efficiently by associating thedetailed information acquirable from the millimeter-wave radar with thecamera image.

Various embodiments have been described above with reference to thedrawings. However, it goes without saying that the present disclosure isnot limited to these embodiments. It is obvious that one of ordinaryskill in the art can conceive various modified examples and correctionexamples within the scope recited in the claims. It should be naturallyunderstood that these modified examples and correction examples belongto the technical scope of the present disclosure. Furthermore, eachcomponent of the above embodiments may be optionally combined withoutdeparting from the gist of the disclosure.

Although examples of configuring the present disclosure by usinghardware have been described in each embodiment described above, thepresent disclosure can also be realized by software in cooperation withhardware.

Each functional block used in the description of the each embodimentdescribed above is realized by an LSI such as an integrated circuittypically. The integrated circuit may control each functional block usedin the description of the each embodiment described above, and mayinclude an input and an output. The functional block may be provided asan individual chip, or a part or all of the functional blocks may beprovided as integrated in a single chip. Reference to the LSI is madehere, but the LSI may be referred to as an IC, a system LSI, a superLSI, or an ultra LSI depending on a difference in the degree ofintegration.

Further, the technique of implementing an integrated circuit is notlimited to the LSI and may be realized by using a dedicated circuit or ageneral-purpose processor. A field programmable gate array (FPGA) thatcan be programmed after the manufacture of the LSI or a reconfigurableprocessor in which the connections and the settings of circuit cellsdisposed inside the LSI can be reconfigured may be used.

If future integrated circuit technology replaces LSIs as a result of theadvancement of semiconductor technology or other derivative technology,the functional blocks could be integrated using another futureintegrated circuit technology. Biotechnology can also be applied.

It should be noted that the present disclosure can be represented as acontrol method performed in a radio communication apparatus or a controlapparatus. Further, the present disclosure can also be represented as aprogram for causing the control method to be operated with a computer.In addition, the present disclosure can be also represented as arecording medium where the program is recorded so as to be readable by acomputer. That is, the present disclosure may be represented in anycategory of devices, methods, programs, and recording media.

It should also be noted that the present disclosure is not limited tothe each embodiment described above in terms of e.g. the type,arrangement, number of members, and alterations can be made asappropriate without departing from the scope of the present inventionby, for example, appropriately substituting the components with thosehaving equivalent operational effects.

The disclosure of Japanese Patent Application No. 2018-072542, filed onApr. 4, 2018, including the specification, drawings and abstract, isincorporated herein by reference in its entirety.

INDUSTRIAL APPLICABILITY

The present disclosure is suitable for use in traffic monitoring.

REFERENCE SIGNS LIST

-   1 Traffic monitoring system-   2 Camera-   3 Millimeter-wave radar-   4 Information processing server (control apparatus)-   5 Remote monitoring PC-   41 Communicator-   42 Data synchronizer-   43 Data accumulator-   44 Screen generator-   45 Mode setter

1. A traffic monitoring system, comprising: a camera that captures amonitoring area including a road, and generates image data; amillimeter-wave radar that scans a scanning area included in themonitoring area, and generates millimeter-wave data; and an informationprocessing server that is connected to the camera and themillimeter-wave radar, and acquires the image data and themillimeter-wave data, wherein the information processing serverincludes: a data synchronizer that synchronizes the image data and themillimeter-wave data such that a difference between a timing at whichthe image data is generated and a timing at which the millimeter-wavedata is generated is equal to or smaller than a certain value; and ascreen generator that associates the image data and the millimeter-wavedata which have been synchronized, with each other, and generates amonitoring screen that indicates a condition of the road.
 2. The trafficmonitoring system according to claim 1, comprising a data accumulatorthat accumulates the image data and the millimeter-wave data which havebeen synchronized by the data synchronizer, wherein the screen generatorgenerates the monitoring screen by using the image data and themillimeter-wave data accumulated in the data accumulator, the image databeing past image data, the millimeter-wave data being pastmillimeter-wave data.
 3. The traffic monitoring system according toclaim 1, wherein: the screen generator performs conversion such thatmillimeter-wave data corresponding to the scanning area corresponds tothe monitoring area.
 4. The traffic monitoring system according to claim1, wherein: the screen generator identifies an object included in thescanning area based on the millimeter-wave data, and superimposesinformation indicating an identification result of the object on theimage data.
 5. The traffic monitoring system according to claim 4,wherein: the information indicating the identification result of theobject includes character information and/or a frame indicating a rangeof the object in the image data.
 6. The traffic monitoring systemaccording to claim 1, wherein: the screen generator determines aretention condition of an object in the scanning area based on themillimeter-wave data, and superimposes information indicating theretention condition on the image data.
 7. The traffic monitoring systemaccording to claim 6, wherein: the information indicating the retentioncondition is color information in accordance with a detection frequencyof the object, and the screen generator superimposes the colorinformation on the image data while making the color informationtransparent on the image data.
 8. The traffic monitoring systemaccording to claim 1, comprising a mode setter that changes a settingrelated to the monitoring screen, the setting being instructed by auser, wherein the screen generator generates the monitoring screen inaccordance with the setting.
 9. A traffic monitoring method, comprising:acquiring image data from a camera, the image data being generated bycapturing a monitoring area including a road; acquiring millimeter-wavedata from a millimeter-wave radar, the millimeter-wave data beinggenerated by scanning a scanning area included in the monitoring area;synchronizing the image data and the millimeter-wave data such that adifference between a timing at which the image data is generated and atiming at which the millimeter-wave data is generated is equal to orsmaller than a certain value; and associating the image data and themillimeter-wave data which have been synchronized, with each other, andgenerating a monitoring screen that indicates a condition of the road.